Until recently cameras for electron microscopes have been manufactured exclusively using an indirect imaging method employing either a scintillator to convert an electron image into a light image, some form of optics to capture the light image and transfer it to a plane offset from the scintillator, and a silicon sensor to capture the light image. The optics have been composed either of a fused fiber optic plate or of a lens. Aside from the primary function of light image transfer, the optics also shield the silicon sensor from direct illumination by the electron beam and from x-rays generated through bremsstrahlung by the beam at the scintillator or higher up in the electron microscope column. This secondary function serves to protect the sensor from damage and to prevent image degradation caused by x-rays hitting the sensor. The optic is only partially effective at performing this latter function.
A small fraction of x-rays generated at the scintillator and higher up in the microscope column make it through the optic and a fraction of those are detected directly in the epitaxial layer of the silicon sensor, creating small bright spots at their point of entry which overlay the image formed by the electron beam.
These spots do not contribute to the quality of the image but, rather, detract from it because 1) the random emission angle at the scintillator and subsequent scattering destroy any spatial correlation of the x-rays with the image-signal-bearing electrons which created them at the scintillator and 2) because the directly detected x-ray adds significantly more signal to the image detected at the silicon sensor than the net signal conferred by the incoming electron after scintillation and image transfer by the optic. Thus it is desirable to minimize the number of these x-ray-generated spots.
The primary method for reducing spots has been to increase the mass lying between scintillator and sensor. This method is not fully effective if either the energy of the electron beam is too high or if there is not sufficient room to place enough material between scintillator and sensor. Furthermore, in the case of lens optics, the optical design will dictate the amount of material which might not be sufficient to provide adequate shielding of x-rays.
In addition, aside from scintillator bremsstrahlung x-rays there are other sources of radiation. They include cosmic rays and decay of radioactive elements in the local environment, which also create bright spots in the image. Shielding by the camera housing and by the optics serves to reduce these forms of radiation as well but, as in the case of bremsstrahlung, this is not 100% effective. In addition, direct detection sensors without a scintillator and transfer optics will have issues with detection of stray scatter and other radiation sources which are not described by the statistics of the specimen image and which are disturbing to the image quality.
Thus there is a need for a technique to completely eliminate the spots created in the sensor by x-rays created in the sensor by the incoming electron beam and by other radiation sources. Spots from various sources which do not conform to the statistical characteristics of the intended image are called outliers. An image processing technique for eliminating outliers would address all of the above various sources of radiation-induced spots without regard to origin.
Image processing techniques based on removal of outliers from an integrated exposure are known. In these techniques, local image statistics are evaluated to establish a local expected range of non-outlier values and outliers are identified as those pixels which do not fall in that local expected range. The range is normally established using a multiple of standard deviations which is chosen to minimize false identification of outliers from pixels which do not suffer from a direct radiation detection event. Since in general, the standard deviation or other measure of statistical deviation varies depending on the local illumination strength (which will be referred to here as electron dose per pixel or electron dose for short) and hence also the local specimen image brightness, and since the specimen can change from one image to the next and from one region to the next within one image, the threshold itself has typically been evaluated separately for each image which is to be processed.
There are a number of problems with this approach. First, it relies on the assumption of ergodicity, i.e. that the statistics of a local group of pixels match the statistics of one of those pixels over a series of acquisitions. This assumption breaks down for high-contrast or rapidly varying specimen images. The effect of specimen contrast can be minimized by evaluating statistics of only a small local region around the pixel being tested but this requirement reduces the number of pixels which are used to calculate the average signal level and of the standard deviation used to generate the outlier threshold criterion. This forces a compromise which works well for low-contrast specimens but still breaks down for more difficult image content such as diffraction patterns or images containing sharp edges, allowing high-contrast features in an image (such as diffraction pattern spots) to be falsely identified as radiation event outliers.
A second problem is that as dose level increases, the histogram of ordinary Gaussian noise in the indirect image (the intended, “good” image from scintillator light generation, optical transfer and sensor detection) grows to obscure more and more of the histogram of the directly detected radiation events. In the case of scintillator bremsstrahlung, which is the most important source of outliers in transmission electron microscope imaging, especially at accelerating voltages above 200 kV, the x-ray outlier count rate is proportional to the beam intensity. This effect is illustrated by FIG. 1, which depicts the result of exposing an indirectly coupled transmission electron microscope camera to a 400 kV uniform beam at two different dose levels of approximately 100 and 7000 beam electrons per detector pixel. At each dose, a pair of identical exposures was taken and differenced to remove fixed pattern gain variations from the histogram. To generate the curves in FIG. 1, the histogram of the absolute value of the difference image was divided by the total average dose in the exposure pair to form a normalized histogram with units: pixels per incoming beam electron at a given count rate. The x-axis of the graph represents the absolute magnitude in counts of a pixel's deviation from the image mean whether high or low. Both the low (100 electron per pixel) dose 1 and the high (7000 electrons per pixel) dose 2 histograms can be seen to consist of two components each: a Gaussian-shaped 3 (inverse parabola in the log display) indirect image noise distribution and a decaying exponential 4 (downward sloping approximately line-shaped in the log display) distribution of x-ray-generated outliers. The normalized noise distribution gets wider and lower as beam intensity goes higher while the normalized distribution of counts at a given pixel per incoming beam electron is constant as predicted for beam-generated bremsstrahlung x-rays in the region of the histogram above the Gaussian distribution. For low dose (100 electrons per pixel) this starts at about 150 counts 5 while for high dose (7000 electrons per pixel) this starts at about 700 counts 6. Thus, at higher doses, a significant portion of the outlier distribution is covered by the Gaussian distribution and can therefore not be discriminated based on an outlier threshold approach. While the argument can be made that if it can't be discriminated it is also not visible and therefore not a problem, outliers near the envelope of the normal distribution can still contribute appreciably to the total noise. In this example, it can be shown that the outliers left in the 7000 electron per pixel image by thresholding at 700 counts instead of 150 counts account for about 5% of the variance in the 7000 electron per pixel power spectrum.
After detection, the simplest method for correction of outliers is to return them to the local mean. An alternative is to set the outlier pixel to a value interpolated from nearest neighbors. For high-contrast images with significant spatial variation, errors can be introduced here as well, in that average-setting and interpolation will both be inaccurate if the image is too rapidly varying as a function of position.
Therefore there is a need for an improved method for providing image outlier detection and correction.